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Abstract 

This  Data  Management  Plan  (DMP)  details  the  Remote  Sensing  dataset 
which  is  maintained  at  the  U.S.  Army  Corps  of  Engineers  (USACE) 
Engineer  Research  and  Development  Center’s  Coastal  and  Hydraulics 
Laboratory  (ERDC-CHL)  Field  Research  Facility  (FRF).  Information  is 
organized  within  the  following  categories:  General  Description,  Points  of 
Contact,  Data  Stewardship,  Data  Documentation,  Data  Sharing,  Initial 
Data  Storage  and  Protection,  Long-Term  Archiving  and  Preservation, 
Hardware  and  Software  Requirements,  Products/Programs,  Tools,  Data 
Catalog,  and  Abbreviations  and  Acronyms. 


DISCLAIMER:  The  contents  of  this  report  are  not  to  be  used  for  advertising,  publication,  or  promotional  purposes. 
Citation  of  trade  names  does  not  constitute  an  official  endorsement  or  approval  of  the  use  of  such  commercial  products. 
All  product  names  and  trademarks  cited  are  the  property  of  their  respective  owners.  The  findings  of  this  report  are  not  to 
be  construed  as  an  official  Department  of  the  Army  position  unless  so  designated  by  other  authorized  documents. 

DESTROY  THIS  REPORT  WHEN  NO  LONGER  NEEDED.  DO  NOT  RETURN  IT  TO  THE  ORIGINATOR. 
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Preface 

This  study  was  conducted  for  the  U.S.  Army  Corps  of  Engineers  (USACE) 
under  the  Coastal  Ocean  Data  Systems  (CODS)  Program,  Work  Unit 
E170H6B,  "Field  Research  Facility  Data  Integration  Framework."  The 
technical  monitor  was  Dr.  J.  P.  Waters. 

The  work  was  performed  by  the  Coastal  Observations  and  Analysis  Branch 
(CEERD-HFA)  of  the  Flood  and  Storm  Protection  Division  (CEERD-HF), 
U.S.  Army  Engineer  Research  and  Development  Center,  Coastal  and 
Hydraulics  Laboratory  (ERDC-CHL).  At  the  time  of  publication,  Dr.  J.  P. 
Waters  was  Chief,  CEERD-HFA,  and  Dr.  T.  Wamsley  was  Chief,  CEERD- 
HF.  The  Director  of  CHL  was  Jose  E.  Sanchez. 

The  USACE  CODS  Program  is  administered  at  CHL  under  the  USACE 
Navigation  Research,  Development,  and  Technology  Transfer  (RD&T) 
Program.  At  the  time  this  effort  was  conducted,  Jeffrey  A.  McKee  was  the 
Headquarters,  USACE  (HQUSACE)  Navigation  Business  Line  Manager 
overseeing  the  CODS  Program.  W.  Jeff  Lillycrop,  CHL,  was  the  ERDC 
Technical  Director  for  Civil  Works  and  Navigation.  Dr.  Waters  was  the 
USACE  CODS  Program  Manager. 

COL  Bryan  S.  Green  was  Commander  of  ERDC,  and  Dr.  Jeffery  P.  Holland 
was  the  Director. 
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Abbreviations  and  Acronyms 


AMD 

advanced  micro  devices 

AOI 

area  of  interest 

ASA 

(See  RPS  ASA) 

asc 

ASCII  (file  format) 

ASCII 

American  Standard  Code  for  Information  Interchange 

BIN 

binary  (file  format) 

CEERD-HF 

U.S.  Army  Corps  of  Engineers,  Engineer  Research  and 
Development  Center,  Flood  and  Storm  Protection 
Division 

CEERD-HF-A 

U.S.  Army  Corps  of  Engineers,  Engineer  Research  and 
Development  Center,  Flood  and  Storm  Protection 
Division,  Coastal  Observations  and  Analysis  Branch 

CESAM 

U.S.  Army  Corps  of  Engineers,  Mobile  District 

CESAM-OP-J 

U.S.  Army  Corps  of  Engineers,  Mobile  District, 
Operations  Division,  Spatial  Data  Branch 

CHL 

Coastal  and  Hydraulics  Laboratory 

CHS 

Coastal  Hazards  System 

CLARIS 

Coastal  Lidar  and  Radar  Imaging  System 

COAB 

Coastal  Observation  and  Analysis  Branch 

CORS 

Continuously  Operating  Reference  Station 

CRREL 

Cold  Regions  Research  and  Engineering  Laboratory 

CS-W 

Catalog  Service  for  the  Web 

DEM 

digital  elevation  model 

DMZ 

demilitarized  zone 

DSM 

digital  surface  model 

DTM 

digital  terrain  model 

EO 

electro-optical 

ERDC 

U.S.  Army  Engineer  Research  and  Development  Center 
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ERDC-CHL 

U.S.  Army  Engineer  Research  and  Development  Center, 
Coastal  and  Hydraulics  Laboratory 

ESRI 

Environmental  Systems  Research  Institute 

FDIF 

Field  Research  Facility  Data  Integration  Framework 

FGDC 

Federal  Geographic  Data  Committee 

FRF 

Field  Research  Facility 

FTP 

file  transfer  protocol 

GAMS 

GPS  Azimuth  Measurement  System 

GB 

gigabyte 

GHz 

gigahertz 

GNSS 

Global  Navigation  Satellite  System 

GPS 

global  positioning  system 

GRASS 

Geographic  Resources  Analysis  Support  System 

grd 

grid  (file  format) 

GRiD 

Geospatial  Repository  and  Data 

ICP 

iterative  closest  point 

IMU 

inertial  measurement  unit 

INS 

inertial  navigation  system 

IR 

infrared 

ISO 

International  Organization  for  Standardization 

JPG 

joint  photographic  experts  group  (file  format) 

km 

kilometer 

LAS 

laser  (file  format) 

LAX 

laser  zip  (file  format) 

ill 

meters 

MATLAB 

Matrix  Laboratory 

MB 

megabyte 

NGS 

National  Geodetic  Survey 

NOAA 

National  Oceanic  and  Atmospheric  Administration 

OGC 

Open  Geospatial  Consortium 
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PII 

personally  identifiable  information 

png 

portable  network  graphics  (file  format) 

POC 

point  of  contact 

POSLV 

Position  and  Orientation  System  LV 

QA/QC 

quality  assurance/quality  control 

QT  Modeler 

Quick  Terrain  Modeler 

R&D 

research  and  development 

RAM 

random  access  memory 

rpp 

Riegl  proprietary  file  format 

RPS  ASA 

RPS  Applied  Science  Associates 

RTK-GPS 

Real-Time  Kinematic  Global  Positioning  System 

rxp 

Riegl  extended  Packets  (file  format) 

SBET 

smoothed  best  estimate  of  trajectory 

TBD 

to  be  determined 

TR 

technical  report 

UI 

user  interface 

USACE 

U.S.  Army  Corps  of  Engineers 

UTM 

Universal  Transverse  Mercator 

WIS 

Wave  Information  Studies 
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1  General  Description 

The  U.S.  Army  Engineer  Research  and  Development  Center,  Coastal  and 
Hydraulics  Laboratory  (ERDC-CHL)  operates  the  Field  Research  Facility 
(FRF),  an  internationally  recognized  coastal  observatory  established  in 
1977  and  located  on  the  Atlantic  Ocean  near  Duck,  NC.  The  ERDC-CHL 
Coastal  Observations  and  Analysis  Branch  (COAB)  maintains  a  number  of 
remotely  sensed  datasets,  including  mobile  and  stationary  lidar,  X-band 
radar,  and  electro-optical  (EO)  and  infrared  (IR)  imagery  at  the  FRF. 

These  datasets  provide  near  real-time  observations  of  the  littoral  zone 
throughout  the  year  as  well  as  during  storm  events.  They  have  become  an 
important  national  resource  for  the  coastal  engineering  and  scientific 
communities  as  a  means  of  providing  necessary  in  situ  data  for  numerical 
model  development,  calibration,  and  verification. 

To  protect  and  preserve  this  important  national  archive,  COAB  has  been 
charged  with  developing  a  Field  Research  Facility  Data  Integration 
Framework  (FDIF),  providing  a  modern,  discoverable  archive  with  easily 
accessible  information  for  strategic  analysis  and  reporting.  To  accomplish 
this,  COAB  has  partnered  with  the  USACE  Mobile  District,  Spatial  Data 
Branch  (CESAM-OP-J)  and  Applied  Science  Associates  (RPS  ASA).  The 
ERDC-CHL  Wave  Information  Studies  (WIS)  and  Coastal  Hazards  System 
(CHS)  will  also  be  incorporated  into  the  framework.  It  is  intended  that  the 
results  of  this  project  will  provide  a  framework  that  can  be  utilized  by  other 
data  management  groups  within  both  ERDC-CHL  and  its  District  partners. 

This  plan  addresses  the  management  of  remote  sensing  data  within  the 
FDIF.  Additional  plans  will  be  prepared  simultaneously  for  the  manage¬ 
ment  of  survey  and  oceanographic  data.  The  project  is  being  conducted  in 
multiple  phases.  Phase  I  addresses  mobile  terrestrial  lidar  datasets  collected 
between  2011  and  2013.  Phase  II  addresses  datasets  collected  during  and 
after  2014  along  with  stationary  lidar  and  mobile  radar  data  collections. 

This  plan  addresses  the  management  of  only  the  Phase  I  datasets.  It  will  be 
updated  once  Phase  II  is  in  operation  and  released  as  a  separate  report. 

1.1  Dataset  name 

Field  Research  Facility  Remote  Sensing  Data 
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1.2  Data  keywords 

•  bathymetry  inversion 

•  Field  Research  Facility 

•  FRF 

•  dune 

•  foredune 

•  beach 

•  foreshore 

•  subaerial 

•  CLARIS 

•  LAS/LAZ 

•  lidar 

•  point  cloud 

•  profile 

•  sand  bar 

•  surf  zone  morphology 

•  survey 

•  topography 

•  digital  elevation  model 

•  DEM/DSM/DTM 

•  radar 

1.3  Data  description 

The  initial  remote  sensing  dataset  includes  beach  topography  collected 
from  a  terrestrial  lidar  scanner  on  both  a  mobile  (CLARIS— Coastal  Lidar 
And  Radar  Imaging  System)  and  stationary  platform.  The  data  is  from 
site-specific  project  sites;  recurring  bi-monthly  surveys  along  the  town  of 
Duck,  NC;  and  storm  surveys  conducted  before,  during,  and  after  storms. 
Terrestrial  lidar  data  generates  x,y,z  point  clouds,  which  are  classified  and 
edited.  Bare  earth  points  are  then  gridded  into  digital  elevation  models 
(DEMs)  to  generate  topographic  surfaces. 

1.4  Data  temporal  extent 

This  dataset  includes  data  collected  intermittently  from  2011  to  2013.  Each 
collection  typically  comprises  2-4  hours  of  surveying  time  and  is  usually 
centered  around  low  tide  for  imaging  the  beach  at  its  widest,  most  exposed 
state. 
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1.5  Data  geographic  extent 

This  dataset  includes  various  coastal  areas  along  the  East  Coast  of  the 
United  States,  including  Fire  Island,  NY,  and  Oak  Island  and  Bald  Head 
Island,  NC.  However,  the  majority  of  the  data  was  collected  near  USACE’s 
Field  Research  Facility  in  Duck,  NC  (highlighted  in  red,  Figure  l). 


Figure  1.  Geographic  extent  of  mobile  terrestrial  lidar  data  collected  at  Duck,  NC. 
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1.6  Data  types 

Lidar  data  products  include  RXP  binary  raw  files,  rectified  point  clouds 
(.las/.laz),  and  gridded  surface  digital  elevation  model  (ASCII)  files,  as 
shown  in  Table  l. 
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Table  1.  Remote  sensing  data  types. 


Data  Type 

Data  Host 

Expected  FDIF  Ingestion 

Mobile  Ildar 

.las/.laz  =  Geospatial 
Repository  &  Data  (GRiD) 

DEMs  =  Mobile  District  DMZ 

End  of  November  2014 

Stationary  lidar 

DEMs  =  Mobile  District  DMZ 

July  2015 

Mobile  radar 

GRiD  [TBD] 

TBD 

EO/IR  imagery 

GRiD  [TBD] 

Summer  2016 

Raw  data  (.rxp,  .rpp,  .bin) 

GRiD  [TBD] 

TBD 

1.7  Data  capture/creation  method 

Lidar  data  was  captured  by  a  Riegl  VZ-iooo/z39oi  terrestrial  lidar  scanner 
system  with  an  Applanix  POS  LV  inertial  navigation  system  (INS). 

1.8  Data  volume 

Approximately  220  gigabytes  (GB)  of  mobile  terrestrial  lidar  data  will  be 
uploaded  to  the  Geospatial  Repository  &  Data  (GRiD)  database.  Smaller 
DEM  products  will  be  derived  from  these  data  and  uploaded  to  the  Mobile 
District  demilitarized  zone  (DMZ)  for  visualization  within  the  FDIF  user 
interface.  These  products  include  full-survey-extent  DEMs  broken  up  into 
tiles  of  smaller  size  for  more  efficient  downloading  and  processing  time. 
Full-extent  DEMs  are  1-2  GB  each,  with  each  tile  at  approximately  too 
MB.  Stationary  lidar  DEMs  are  approximately  25-50  MB  each.  The 
workflow  for  DEM  generation  is  in  the  final  stages  of  development,  and 
more  precise  data  size  requirements  will  be  available. 

1.9  Data  restrictions 

No  personally  identifiable  information  (PII)  or  other  information  whose 
distribution  may  be  restricted  by  law  or  national  security  is  included  in 
this  dataset. 

1.10  Plan  storage 

This  plan  will  be  stored  at  the  USACE  FRF  in  Duck,  NC,  and  at  the  ERDC 
in  Vicksburg,  MS. 
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2  Points  of  Contact  (POC) 

2.1  Project  POC 

Katherine  L.  Brodie,  Research  Oceanographer,  USACE  Field  Research 
Facility,  1261  Duck  Road,  Kitty  Hawk,  NC  27949;  Katherine.L.Brodie@usace.armv.mil. 
(252)  261-6840  X233. 

Nicholas  J.  Spore,  Research  Civil  Engineer,  USACE  Field  Research 
Facility,  1261  Duck  Road,  Kitty  Hawk,  NC  27949;  Nicholas.J.Spore@usace.armv.mil. 
(252)  261-6840x231. 

2.2  Overall  POC 

Jeffrey  P.  Waters,  Chief,  Coastal  Observations  and  Analysis  Branch, 

USACE  Field  Research  Facility,  1261  Duck  Road,  Kitty  Hawk,  NC  27949; 
Jeffrev.P.Waters@usace.armv.mil.  (252)  261-6840  X233- 

2.3  Data  Quality  POC 

Nicholas  J.  Spore,  Research  Civil  Engineer,  USACE  Field  Research 
Facility,  1261  Duck  Road,  Kitty  Hawk,  NC  27949;  Nicholas.J.Spore@usace.armv.mil. 
(252)  261-6840x231. 

2.4  Data  Collection  POC 

Nicholas  J.  Spore,  Research  Civil  Engineer,  USACE  Field  Research 
Facility,  1261  Duck  Road,  Kitty  Hawk,  NC  27949;  Nicholas.J.Soore@usace.armv.mil. 
(252)  261-6840x231. 

2.5  Documentation/Metadata  POC 

Nicholas  J.  Spore,  Research  Civil  Engineer,  USACE  Field  Research 
Facility,  1261  Duck  Road,  Kitty  Hawk,  NC  27949;  Nicholas.J.Soore@usace.armv.mil. 
(252)  261-6840x231. 

Data  Storage/ Disaster  Recovery  POC 

Nicholas  J.  Spore,  Research  Civil  Engineer,  USACE  Field  Research 
Facility,  1261  Duck  Road,  Kitty  Hawk,  NC  27949;  Nicholas.J.Soore@usace.armv.mil. 
(252)  261-6840x231. 


2.6 
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2.7  Data  Management  Plan  (DMP)  Adherence/Implementation  POC 

Mike  Forte,  Research  Physical  Scientist,  USACE  Field  Research  Facility, 
1261  Duck  Road,  Kitty  Hawk,  NC  27949;  Michael.F.Forte@usace.armv.mil.  (252) 
261-6840  X228. 
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3  Data  Stewardship 

3.1  Quality  control 

3.1.1  Mobile  lidar 

The  following  quality  assurance/quality  control  (QA/QC)  procedures  are 
followed  when  validating  the  integrity  of  terrestrial  lidar  data  due  to  its 
structure  and  large  spatial  domain.  The  trajectory  information  used  to 
rectify  the  point  data  is  first  analyzed  for  obvious  faults,  satellite  drop¬ 
outs,  global  positioning  system  (GPS)  Azimuth  Measurement  Subsystem 
(GAMS)  ambiguities,  and  positional  error  root  mean  square  (RMS)  for 
latitude,  longitude,  and  elevation  through  the  entire  trajectory.  The 
rectified,  georeferenced  point  cloud  is  then  compared  to  previously 
validated  surveys  along  fixed  structures,  such  as  rooflines  and  road 
centerlines,  to  quantify  the  survey  accuracy  and  repeatability.  Control 
points  positioned  along  the  trajectory  that  were  surveyed  with  real-time 
kinematic  (RTK)-GPS  are  also  used  as  an  accuracy  check.  In  addition, 
transects  are  extracted  perpendicular  to  the  trajectory  to  measure 
temporal  variance  due  to  changes  in  the  satellite  constellation  and  GPS 
signal  quality  along  the  survey  route,  and  overlapping  regions  are 
compared  for  vertical  closure. 

3.1.2  Stationary  lidar 

For  the  stationary  lidar  scans,  QA/QC  flags  are  implemented  into  the 
processing  workflow.  Iterative  closest  point  (ICP)  algorithms  are  applied 
to  adjust  each  successive  scan’s  roll,  pitch,  and  yaw  relative  to  a  stable  and 
verified  baseline  survey.  After  the  ICP  adjustment  is  performed,  an 
indicator  flag  that  represents  the  quality  of  the  adjustment  is  attached  to 
the  metadata  and  is  used  to  pass  acceptable  scans  further  into  the 
processing  regime  or  less-than-acceptable  scans  passed  to  the  user  for 
visual  inspection. 

3.2  Data  lifecycle 

3.2.1  Mobile  lidar 

Terrestrial  GPS  data  is  postprocessed  with  either  National  Geodetic 
Survey’s  (NGS)  Continuously  Operating  Reference  Stations  (CORS)  or  a 
static,  local  base  station  and  integrated  with  postprocessed  inertial 
measurement  unit  (IMU)  data  through  Applanix’s  POSPac  Global 
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Navigation  Satellite  System  (GNSS)  software  to  derive  a  smoothed  best 
estimate  of  trajectory  (SBET).  After  processing,  quantitative  quality 
metrics  are  analyzed  to  ensure  the  trajectory  accuracy  meets  the 
manufacturer’s  specified  values. 

The  SBET  is  imported  into  Riegl’s  RiProcess  lidar  software  program, 
where  the  lidar  point  cloud  is  rectified  and  georegistered  to  a  particular 
coordinate  system,  projection,  and  geoid  model,  and  boresight  values  for 
roll,  pitch,  and  yaw  are  applied.  Control  monuments  and  previously 
validated  datasets  are  used  for  comparison  to  determine  the  accuracy  of 
the  survey.  These  are  measured  by  cutting  cross  sections  of  the  current 
survey  overlaid  on  previously  validated  data  within  the  software  through 
stable  structures,  such  as  houses,  monuments,  and  road  centerlines. 

The  processed  point  cloud  is  manipulated  within  RiProcess  to  delineate 
the  shoreline,  filter  structures,  and  vegetation,  and  to  classify  points  as 
American  Society  for  Photogrammetry  and  Remote  Sensing  (ASPRS) 
classification  standards  for  water,  vegetation,  structures,  and  ground.  The 
classified  point  cloud  is  exported  as  a  .las(i.i)  file,  which  is  then  ingested 
into  the  workflow  described  in  Figure  2,  where  it  is  gridded  into  a  DEM. 
The  DEM  is  created  with  the  Points2Grid  algorithm  utilized  by  USACE’s 
GRiD  geospatial  archiving  system  at  0.5  m  resolution  at  mean  elevation 
within  each  grid  cell. 

3.2.2  Stationary  lidar 

The  stationary  lidar  data  lifecycle  is  still  in  development  and  follows  a 
different  workflow.  After  the  initial  collection,  data  is  coregistered  to  a 
baseline  survey  using  carefully  selected  permanent  structures  (i.e.,  control 
regions)  for  either  ICP  or  plane  fitting/matching  algorithms.  Afterwards, 
the  point  cloud  is  passed  through  noise,  range,  and  classification  filters  to 
correctly  trim  and  classify  the  points.  Bare  earth  points  are  then  saved  as  a 
.laz  file  and  gridded  into  a  DEM  using  the  same  process  as  used  for  mobile 
data  (Figure  2). 
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Figure  2.  Workflow  for  continuous  stationary  lidar  data  products. 
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4  Data  Documentation 

4.1  Metadata  repository 

The  Environmental  Systems  Research  Institute’s  (ESRI)  Geoportal  Server, 
which  has  been  set  up  by  CESAM-OP-J,  is  used  as  the  FDIF  metadata 
repository.  Geoportal  is  a  free,  open-source  product  for  managing  and 
publishing  metadata.  It  facilitates  discovery  of  resources,  including 
datasets  and  web  services. 

Geoportal  Server  can  be  used  to  create  a  Geoportal  website,  where  data 
managers  can  register  data  resources  for  discovery  by  users.  Geoportal 
stores  and  catalogs  metadata  and  access  information  for  each  registered 
data  resource.  All  metadata  is  contained  in  an  Open  Geospatial 
Consortium  (OGC)-compliant  geoportal  CS-W  2.0.2  catalog  service. 

4.2  Additional  metadata 

Additional  raw  field  metadata  can  be  provided  via  scanned  field  log  books 
that  can  be  linked  within  the  original  metadata  files.  The  feasibility  of 
using  these  scanned  images,  as  well  as  where  they  would  be  stored,  is  still 
yet  to  be  determined. 

Hourly  metadata,  which  is  automatically  generated  after  each  collection, 
accompanies  each  hourly  stationary  lidar  scan.  This  metadata  provides  all 
parameters  used  for  each  collection,  including  all  postprocessing 
information. 

4.3  Data/Metadata  standards 

In  an  effort  to  catalog  and  properly  manage  multiple  types  of  remote 
sensing  datasets,  detailed  metadata  files  are  included  within  each  location 
or  project-specific  data  collection.  This  promotes  easily  recognized  search 
queries  by  users  from  a  multitude  of  agencies,  backgrounds,  or  professions 
who  are  interested  in  particular  types  of  data.  The  current  International 
Standards  Organization  (ISO)  metadata  standards  have  been  adopted  as 
the  standard  format  for  all  metadata  files,  and  any  existing  non-ISO 
compliant  files  have  been  updated  to  meet  the  ISO  19115  metadata 
standard.  CatMDEdit— the  software  package  used  for  metadata  editing, 
template  building,  and  populating— promotes  efficiency  with  regard  to 
user  input,  adaptability,  and  conversion  of  older  Federal  Geographic  Data 
Community  (FGDC)  standards  into  the  current  ISO  standards. 


ERDC/CHL  SR-16-2 


11 


5  Data  Sharing 

5.1  Data  availability 

Access  to  this  dataset  is  currently  restricted  to  collaborators  until  research 
goals  are  completed.  Data  is  routinely  collected  bimonthly  with  additional 
extreme  events  captured  when  feasible.  Data  will  be  generally  available  on 
the  FDIF  after  research  goals  have  been  satisfied. 

5.2  Data  availability  restrictions 

This  dataset  is  not  currently  available  to  the  public  because  it  is  stored  on 
an  FRF  Research  and  Development  (R&D)  server,  which  has  no  public 
access.  Recently  collected  data  will  not  be  available  until  specific  research 
goals  are  completed  and  published.  Restricted  access  privileges  may  be 
provided  to  collaborators  and  contributors  on  a  case  by  case  basis. 

5.3  Data  access  restrictions 

Full  acknowledgement  of  observational  and  modeling  data  supplied  by  the 
FRF  provides  an  important  metric  to  assess,  and  ultimately  continue,  its 
service  to  the  coastal  research  community. 

The  following  citation  should  be  included  in  any  use  of  these  data: 

Data  are  provided  by  the  Coastal  and  Hydraulics  Laboratory,  U.S. 

Army  Corps  of  Engineers,  Field  Research  Facility,  Duck,  North 
Carolina. 

Specific  investigators  and/or  relevant  research  programs  should  also  be 
acknowledged  for  their  contributions. 

5.4  Data  access  protocols 

All  data  processed  by  the  FRF  Survey  Application  will  be  accessible  from 
an  enterprise  database  through  the  use  of  standardized  web  mapping 
services.  At  a  minimum,  the  survey  points  (SurveyPoints),  extent  of  survey 
polygons  (SurveyJob),  and  raster  grid  surface  will  be  accessible. 


ERDC/CHL  SR-16-2 


12 


5.5  Catalogue  registration 

These  data  will  be  registered  through  a  geoplatform  available  at 
http://geoplatform.usace.armv.mil/. 
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6  Initial  Data  Storage  and  Protection 

6.1  Initial  data  storage  location 

Data  is  currently  stored  at  the  FRF  in  Duck,  NC;  however,  the  goal  is  to 
store  it  in  a  database  in  the  USACE  CESAM  DMZ,  in  Mobile,  AL,  with  a 
separate  schema  to  solely  support  the  CHL.  The  CESAM-OP-J  will  serve  as 
the  database  administrator. 

6.2  Data  storage  protection 

Data  protection  is  provided  by  a  firewall  between  the  R&D  server  and  the 
general  public  and  through  an  offsite  mirror. 

6.3  Data  access  protection 

Access  to  restricted  data  will  be  managed  via  password.  Additional  details 
will  be  provided  if  and  when  restricted  data  is  designated  to  be  included  in 
the  FDIF  system.  Otherwise,  there  are  no  limitations  to  data  access. 
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7  Long-Term  Archiving  and  Preservation 

7.1  Data  archiving  location 

Long-term  archival  is  planned  for  the  USACE  Cold  Regions  Research  and 
Engineering  Laboratory  (CRREL)  GRiD  system.  This  location  has  been 
notified  and  fully  supports  ingesting  mobile  and  static  lidar  point  clouds  in 
FY15  with  continued  support  for  other  remote  sensing  products  in 
subsequent  years.  The  USACE  Mobile  District  (CESAM)  will  provide 
additional  archiving  on  CESAM  DMZ  servers  of  DEMs  generated  by  the 
point  clouds  hosted  on  GRiD  for  use  within  the  FDIF  portal  for  calculating 
volumes,  slopes,  profile  comparisons,  and  other  measurements. 

7.2  Data  archiving  strategy 

The  immediate  archiving  strategy  focuses  on  off-site  data  hosting  by 
CRREL’s  GRiD  system  and  CESAM’s  DMZ  servers.  Due  to  the  high  volume 
and  continuous  collection  regime  at  the  FRF,  the  feasibility  of  setting  up 
an  instance  of  GRiD  locally  at  the  facility  in  Duck,  NC,  is  currently  being 
investigated.  This  would  allow  for  faster  and  more  efficient  data  uploading 
as  well  as  reducing  bandwidth  limitations  when  accessing  large  amounts  of 
data  for  internal  research  interests. 

7.3  Data  archiving  costs 

Costs  of  long-term  data  archiving  will  be  provided  and  maintained  by  the 
USACE  Coastal  Field  Data  Collection  Program. 

7.4  Data  archiving  procedures 

In  addition  to  the  data  files,  data  descriptions  and  metadata  will  be 
submitted  to  the  archive  to  assist  those  using  the  data  in  the  future.  The 
data  will  be  sent  to  CRREL  (GRiD)  via  external  hard  disk  drives  containing 
point  cloud  data. 

The  DEMs  for  all  of  the  mobile  lidar  data  will  initially  be  sent  to  CESAM 
(DMZ)  via  external  hard  disk  drives.  Subsequent  mobile  data  and 
continuous  stationary  data  will  be  pushed  from  a  local  directory  that  is 
continuously  monitored  for  new  file  updates.  After  proper  QA/QC  is 
completed,  the  new  files  will  be  added  to  this  directory  and  will  be 
available  for  upload  to  the  Mobile  DMZ  servers. 
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7.5  Data  retention  period 

Due  the  importance  of  collecting  and  maintaining  a  long-term 
morphological  record,  the  data  retention  period  is  indefinite. 
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8  Hardware  and  Software  Requirements 

8.1  Hardware  requirements 

Due  to  the  large  amount  of  point  data  stored  within  each  dataset  and  the 
resources  required  to  visualize  a  point  cloud  with  over  400  million  points, 
the  user  should  have  a  computer  with  at  least  15  GB  of  free  hard  disk 
space,  a  minimum  of  8  GB  of  RAM  (16  GB  for  faster  visualization),  and  a 
multicore  Intel  or  AMD  processor  operating  at  2  gigahertz  or  faster. 

8.2  Software  requirements 

8.2.1  Server  software  requirements 

Server  software  requirements  will  be  specified  by  CRREL. 

8.2.2  Client-side  software  requirements 

•  Web  browser  (such  as  Chrome  or  Firefox) 

•  Data  visualization  program  for  point  clouds  and  surfaces  (such  as 
QT  Modeler,  ArcMap,  or  Geographic  Resources  Analysis  Support 
System  [GRASS]) 

•  Data  analysis  tools  of  choice  (such  as  ArcMap,  Matrix  Laboratory 
(MATLAB),  or  Python) 
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9  Products/Programs 

Phase  I  data  products  primarily  focus  on  elevation  data  derived  from  the 
mobile  terrestrial  lidar  surveys  as  well  as  the  near  real-time  stationary 
lidar  surveys.  The  elevation  data  is  represented  in  its  original  form  and 
position  from  the  scanner  as  well  as  interpolated  surface  products.  CRREL 
is  hosting  the  original  point  cloud  data  in  GRiD,  and  CESAM  is  hosting  the 
interpolated  DEMs  used  for  analysis  in  the  CESAM  DMZ. 

Phase  II  data  products  will  potentially  include  more  recent  mobile  lidar 
surveys,  mobile  radar  imagery,  and  EO/IR  imagery. 
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10  Tools 

10.1  Visualization  tools 

The  FDIF  user  interface  (UI)  tools  serve  as  the  primary  visualization  tools 
for  the  FRF  remote  sensing  data.  The  UI  is  a  map-based  view  that  shows 
the  location  of  all  FRF  assets  and  provides  tabular  and  graphical 
summaries  of  the  various  data  types.  Color-shaded  DEMs  for  elevation, 
slope,  aspect,  and  other  data  are  available  with  an  adjustable/configurable 
color  scale  and  gradients.  Transparency  options  for  adjusting  overlay 
opacity  and  a  time  slider  bar  for  available  data  are  also  implemented. 

10.2  Analysis  tools 

10.2.1  Transect  tool 

Users  can  draw  a  transect  or  input  coordinate  end  points  to  extract  profiles 
from  multiple  DEMs  in  the  same  location  over  a  user-specified  time  range. 

User  Inputs 

1.  End  points  of  transect 

•  Click  points  on  the  map 

•  Manually  input  coordinates 

•  Identify  an  FRF  survey  line  number  or  click  a  survey  line 

2.  Dates  of  interest 

•  Default:  most  recent  DEM 

•  Input  a  start  and  end  date  (all  datasets  that  reside  within  that  date 
range) 

•  Input  a  date  interval  (hourly/daily/monthly /yearly) 


Plot  Output:  Refer  to  Figure  3. 
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Figure  3.  Map  and  transect  views  of  three  separate  transects  of  interest. 


Elevation  Change  During  the  Storm 
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Distance  down  transect  (m) 


Data  Download 


•  ASCII  file  containing  a  date,  X,  Y,  distance  along  transect,  and  Z  of  the 
displayed  transect 

•  .jpg  or  .png  of  the  generated  plot 

Future  Products 


•  Multiple-points  definition  of  the  transect  path 

•  Left/right  offset  from  the  initial  transect  given  an  input  of  distance  to 
offset  and  number  of  lines 

•  Transect  elevation  and  change  through  time  as  timestack  (PColor) 
plots,  where  the  representative  transect  is  plotted  as  distance  along  the 
transect  on  the  y-axis  and  time  on  the  x-axis  and  colored  by  elevation 
or  elevation  change 
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10.2.2  Elevation  time  series  tool 

Users  can  pick  a  point  within  the  data  extents  and  specify  a  time  range  to 
plot  the  elevation  as  a  function  of  time  over  all  data  available  for  the  point 
of  interest. 

User  Inputs 

1.  Location  for  the  time  series  (that  is,  the  point  of  interest) 

•  Click  on  the  map 

•  Input  coordinates 

•  Click  multiple  points  for  comparing  different  locations  and  their 
respective  elevation  time-series 

2.  Dates  of  interest 

•  Input  a  start  and  end  date  (all  datasets  that  reside  within  that  date 
range) 

•  Input  a  date  interval  (hourly/daily/monthly /yearly) 

Plot  Output:  Refer  to  Figure  4. 


Figure  4.  Elevation  time  series  at  a  specified  point. 
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Data  Download 


•  ASCII  file  containing  a  date,  X,  Y,  Z,  and  point  number  or  unique  (user- 
specified)  identifier 

•  .jpg  or  .png  of  the  generated  plot 

10.2.3  DEM  visualization/download  tool 

User  can  visualize  a  color-shaded  digital  elevation  model  on  a  map  and 
manipulate  its  representation  for  a  particular  application. 

User  Inputs 

1.  Boundary 

•  Default:  maximum  extent  of  the  data 

•  Draw  a  box  on  the  map  for  an  area  of  interest 

•  Input  bounding  coordinates 

2.  Dates  of  interest 

•  Default:  most  recent  DEM 

•  Input  a  start  and  end  date  (all  datasets  that  reside  within  that  date 
range) 

•  Input  a  date  interval  (hourly/daily/monthly /yearly) 

Plot  Output:  Refer  to  Figure  5. 

Display  Properties 

•  User-defined  color  axes 

•  Ability  to  edit  the  color  map 

•  No-data  values  display  as  transparent 

Data  Download 


•  ASCII  grid  file  of  elevations 

•  .jpg  or  .png  of  the  generated  plot 
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Figure  5.  Color-shaded  elevation  raster  displayed  over  aerial  imagery. 


10.2.4  Surface  difference  tool 

Two  specified  datasets  from  different  collection  dates  but  overlapping 
geographic  extents  can  be  subtracted  with  respect  to  each  other  to  produce 
a  change  map  with  customizable  colors,  color  scale,  and  gradient.  Areas 
that  do  not  geographically  overlap  are  represented  as  ioo%  transparent. 

User  Inputs 

1.  Boundary 

•  Default:  maximum  extent  of  data 

•  Draw  a  box  on  the  map  for  an  area  of  interest 

•  Input  bounding  coordinates 

2.  Dates  of  interest  (two  are  required  for  differencing) 

•  Default:  most  recent  two  dates 

•  Input  a  start  and  end  date  (all  datasets  that  reside  within  that  date 
range) 

•  Input  a  date  interval  (hourly/daily/monthly /yearly) 

•  Plot  Output:  Refer  to  Figure  6. 
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Figure  6.  Color-shaded  surface  elevation  difference  map  between  pre-  and  post-storm  dates. 


Display  Properties 

•  User-defined  color  axes 

•  Ability  to  edit  the  color  map 

•  Areas  of  no  data  or  no  data  overlap  are  transparent 

•  No-data  values  are  transparent 

Data  Download 


•  ASCII  grid  file  of  elevation  differences 

•  .jpg  or  .png  of  the  generated  plot 

10.3  Upload  tools 

FRF  staff  will  upload  new  datasets  to  the  host  site  via  file  transfer  protocol 
(FTP)  or,  when  necessary  due  to  the  size  of  the  dataset  or  a  limited  local 
bandwidth,  delivery  of  an  external  hard  disk  drive. 
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10.4  Download  tools 

10.4.1  Rectangular/Polygon  areas  of  interest  (AOis) 

Users  can  draw  an  AOI  on  the  map  interface  to  display  a  table  that 
identifies  all  data  available  for  download  directly  from  the  FDIF  portal  and 
also  other  datasets  available  within  GRiD. 

10.4.2  Coordinate  system  conversion 

Data  can  be  reprojected  into  a  multitude  of  coordinate  systems  for 
download,  with  initial  emphasis  on  Universal  Transverse  Mercator  (UTM), 
State  Plane  Systems,  Geographic,  and  local  FRF  coordinates. 

10.5  Other  tools 

Depending  upon  FDIF  development  completed  this  year,  additional  tools 
may  be  developed— for  example,  a  tool  to  create  custom  plots  from  blank 
axes  with  options  for  time,  distance  along  transect,  elevation,  volume, 
slope,  and  elevation  change.  The  ability  to  add  oceanographic  data  as  a 
subplot  would  help  analyze  morphologic  changes  with  respect  to  the 
oceanographic/environmental  forcing  (for  example,  a  time  series  of  wave 
height  as  a  subplot  under  a  time  series  of  elevation  change). 
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Appendix  A:  Data  Catalog 

A.l  Mobile  terrestrial  lidar  data  2011-2013  —  Point  clouds 

Description:  High-resolution  lidar  point  cloud  data  (total  220  GB) 


Table  Al.  Mobile  terrestrial  lidar  data  2011-2013  —  Point  clouds. 


Survey  Date 

Scanner 

Type 

Duration/ 

Length 

Classification 

Scheme 

Formats 

Archive 

Size 

Products/Notes 

25  Aug  2011 

Z390i 

11  km 

None 

.las/.laz 

0.80  GB 

Pre-Hurricane  Irene,  Duck,  NC 

28  Aug  2011 

Z390i 

4.5  km 

None 

.las/.laz 

0.50  GB 

Post-Hurricane  Irene,  Duck,  NC 

31  Jan  2012 

VZ1000 

30  km 

Yes 

.las/.laz 

23.9  GB 

Fire  Island,  NY  (winter) 

13  Jun  2012 

VZ1000 

FRF  property 

Yes 

.las/.laz 

0.49  GB 

Summer  beach  conditions 

14  Aug  2012 

VZ1000 

10  km 

Yes 

.las/.laz 

11.3  GB 

Oak  Island,  NC  (summer) 

15  Aug  2012 

VZ1000 

10  km 

Yes 

.las/.laz 

10.4  GB 

Bald  Head  Island,  NC  (summer) 

06  Nov  2012 

VZ1000 

12  km 

Yes 

.las/.laz 

8.75  GB 

Post-Hurricane  Sandy,  Duck,  NC 

06  Jan  2013 

VZ1000 

10  km 

Yes 

.las/.laz 

10.6  GB 

Oak  Island,  NC  (winter) 

07  Jan  2013 

VZ1000 

10  km 

Yes 

.las/.laz 

11.2  GB 

Bald  Head  Island,  NC  (winter) 

06  Mar  2013 

VZ1000 

12  km 

Yes 

.las/.laz 

9.69  GB 

Pre-Nor'easter,  Duck,  NC 

07  Mar  2013 

VZ1000 

12  km 

Yes 

.las/.laz 

6.20  GB 

4  m  Nor'easter,  Duck,  NC 

08  Mar  2013 

VZ1000 

12  km 

Yes 

.las/.laz 

6.13  GB 

4  m  Nor'easter,  Duck,  NC 

09  Mar  2013 

VZ1000 

12  km 

Yes 

.las/.laz 

6.57  GB 

4  m  Nor'easter,  Duck,  NC 

10  Mar  2013 

VZ1000 

12  km 

Yes 

.las/.laz 

7.64  GB 

4  m  Nor'easter,  Duck,  NC 

11  Mar  2013 

VZ1000 

12  km 

Yes 

.las/.laz 

6.61GB 

4  m  Nor'easter,  Duck,  NC 

13  Mar  2013 

VZ1000 

12  km 

Yes 

.las/.laz 

8.82  GB 

4  m  Nor'easter,  Duck,  NC 

20  Mar  2013 

VZ1000 

12  km 

Yes 

.las/.laz 

6.97  GB 

Post-Nor'easter,  Duck,  NC 

10  Apr  2013 

VZ1000 

30  km 

Yes 

.las/.laz 

14.5  GB 

Fire  Island,  NY  (storm) 

11  Apr  2013 

VZ1000 

30  km 

Yes 

.las/.laz 

15.7  GB 

Fire  Island,  NY  (storm) 

12  Apr  2013 

VZ1000 

30  km 

Yes 

.las/.laz 

11.8  GB 

Fire  Island,  NY  (storm) 

13  Apr  2013 

VZ1000 

30  km 

Yes 

.las/.laz 

12.0  GB 

Fire  Island,  NY  (storm) 

17  Jun  2013 

VZ1000 

FRF  property 

None 

.las/.laz 

1.11  GB 

NOAA  tri-lidar  target  flights 

17  Jul  2013 

VZ1000 

10  km 

Yes 

.las/.laz 

7.32  GB 

Oak  Island,  NC  (summer) 

18  Jul  2013 

VZ1000 

10  km 

Yes 

.las/.laz 

4.35  GB 

Bald  Head  Island,  NC  (summer) 

04  Sep  2013 

VZ1000 

10  km 

Yes 

.las/.laz 

6.03  GB 

Bi-monthly  morphology,  Duck,  NC 

12  Dec  2013 

VZ1000 

20  km 

Yes 

.las/.laz 

11.5  GB 

Bi-monthly  morphology,  Duck,  NC 
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A.2  Mobile  terrestrial  lidar  data  2011-2013  —  Digital  elevation 
models 

Description:  Mean  elevation  grid  at  0.5  m  resolution  interpolated  from  the 
point  clouds  referenced  in  A.i 


Table  A2.  Mobile  terrestrial  lidar  data  2011-2013  —  Digital  elevation  models. 


Survey  Date 

Scanner 

Type 

Duration/ 

Length 

Classification 

Scheme 

Formats 

Archive 

Size 

Products/Notes 

25  Aug  2011 

Z390i 

11  km 

None 

.asc/.grd 

TBD 

Pre-Hurricane  Irene,  Duck,  NC 

28  Aug  2011 

Z390I 

4.5  km 

None 

.asc/.grd 

TBD 

Post-Hurricane  Irene,  Duck,  NC 

31  Jan  2012 

VZ1000 

30  km 

Yes 

.asc/.grd 

TBD 

Fire  Island,  NY  (winter) 

13  Jun  2012 

VZ1000 

FRF  property 

Yes 

.asc/.grd 

TBD 

Summer  beach  conditions 

14  Aug  2012 

VZ1000 

10  km 

Yes 

.asc/.grd 

TBD 

Oak  Island,  NC  (summer) 

15  Aug  2012 

VZ1000 

10  km 

Yes 

.asc/.grd 

TBD 

Bald  Head  Island,  NC  (summer) 

06  Nov  2012 

VZ1000 

12  km 

Yes 

.asc/.grd 

TBD 

Post-Hurricane  Sandy,  Duck  NC 

06  Jan  2013 

VZ1000 

10  km 

Yes 

.asc/.grd 

TBD 

Oak  Island,  NC  (winter) 

07  Jan  2013 

VZ1000 

10  km 

Yes 

.asc/.grd 

TBD 

Bald  Head  Island,  NC  (winter) 

06  Mar  2013 

VZ1000 

12  km 

Yes 

.asc/.grd 

TBD 

Pre-Nor'easter,  Duck,  NC 

07  Mar  2013 

VZ1000 

12  km 

Yes 

.asc/.grd 

TBD 

4  m  Nor'easter,  Duck,  NC 

08  Mar  2013 

VZ1000 

12  km 

Yes 

.asc/.grd 

TBD 

4  m  Nor'easter,  Duck,  NC 

09  Mar  2013 

VZ1000 

12  km 

Yes 

.asc/.grd 

TBD 

4  m  Nor'easter,  Duck,  NC 

10  Mar  2013 

VZ1000 

12  km 

Yes 

.asc/.grd 

TBD 

4  m  Nor'easter,  Duck,  NC 

11  Mar  2013 

VZ1000 

12  km 

Yes 

.asc/.grd 

TBD 

4  m  Nor'easter,  Duck,  NC 

13  Mar  2013 

VZ1000 

12  km 

Yes 

.asc/.grd 

TBD 

4  m  Nor'easter,  Duck,  NC 

20  Mar  2013 

VZ1000 

12  km 

Yes 

.asc/.grd 

TBD 

Post-Nor'easter,  Duck,  NC 

10  Apr  2013 

VZ1000 

30  km 

Yes 

.asc/.grd 

TBD 

Fire  Island,  NY  (storm) 

11  Apr  2013 

VZ1000 

30  km 

Yes 

.asc/.grd 

TBD 

Fire  Island,  NY  (storm) 

12  Apr  2013 

VZ1000 

30  km 

Yes 

.asc/.grd 

TBD 

Fire  Island,  NY  (storm) 

13  Apr  2013 

VZ1000 

30  km 

Yes 

.asc/.grd 

TBD 

Fire  Island,  NY  (storm) 

17  Jun  2013 

VZ1000 

FRF  property 

None 

.asc/.grd 

TBD 

NOAA  tri-lidar  target  flights 

17  Jul  2013 

VZ1000 

10  km 

Yes 

.asc/.grd 

TBD 

Oak  Island,  NC  (summer) 

18  Jul  2013 

VZ1000 

10  km 

Yes 

.asc/.grd 

TBD 

Bald  Head  Island,  NC  (summer) 

04  Sep  2013 

VZ1000 

10  km 

Yes 

.asc/.grd 

TBD 

Bi-monthly  morphology,  Duck,  NC 

12  Dec  2013 

VZ1000 

20  km 

Yes 

.asc/.grd 

TBD 

Bi-monthly  morphology,  Duck,  NC 
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A.3  Continuous  tower-mounted  stationary  lidar  -  Digital  elevation 
models 

Description:  Mean  elevation  grid  at  0.5  m  resolution  interpolated  from 
hourly  framescan  point  clouds 


Table  A3.  Continuous  tower-mounted  stationary  lidar  data  —  Digital  elevation  models. 


Survey  Date 

Scanner  Type 

Duration/ 

Length 

Classification 

Scheme 

Formats 

Archive 

Size 

Products/Notes 

TBD 

VZ1000/2000 

500  m 

Yes 

.asc/.grd 

TBD 

Hourly  beach  framescans 
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